Monitoring device, monitoring method, monitoring program, and monitoring system

ABSTRACT

A monitoring system includes a mobile terminal, and a management device which communicates with the mobile terminal, in which the mobile terminal includes a first transmission unit which transmits position information of the mobile terminal to the management device at a predetermined interval, and the management device includes a control unit which controls, when acquiring the position information, a direction of a camera based on the position information. A monitoring system may include a mobile terminal and a management device which communicates with the mobile terminal, in which the mobile terminal includes a first transmission unit which transmits position information of the mobile terminal to the management device, and the management device includes a control unit which controls, when acquiring the position information, a direction of a camera based on the position information, a tracking unit which extracts a tracking target candidate based on an image acquired from the camera, and a display unit which displays the image and highlights the tracking target candidate included in the image so as to be selectable.

TECHNICAL FIELD

The present invention relates to a monitoring device, a monitoringmethod, a monitoring program, a monitoring system, and a facilitymonitoring system.

BACKGROUND

In the above technical field, Patent Literature 1 discloses a techniquefor transmitting, when it is detected that a user using a mobilecommunication terminal is in an unusual situation, the user's positioninformation to a monitoring camera, and transmitting an image includingthe user acquired by the monitoring camera to a control center.

CITATION LIST Patent Literature

PTL 1: Japanese Patent Application Laid-Open No. 2004-328333

SUMMARY OF INVENTION Technical Problem

Here, in the above-mentioned Patent Literature 1, the user of the mobilecommunication terminal is a monitoring target, and the situation andcondition of the user can be detected. However, the above-mentionedPatent Literature 1 is based on the assumption that the user of themobile communication terminal does not move. Therefore, when the user ofthe mobile communication terminal is moving, if the camera is controlledto the position acquired from the mobile communication terminal andphotographing is performed, the photographed image cannot necessarilyinclude the user.

A purpose of the present invention is to provide a monitoring device, amonitoring method, a monitoring program, a monitoring system, and afacility monitoring system for solving the above problem.

Solution to Problem

A monitoring system according to the present invention includes a mobileterminal, and a management device which communicates with the mobileterminal, in which the mobile terminal includes a first transmissionunit which transmits position information of the mobile terminal to themanagement device at a predetermined interval, and the management deviceincludes a control unit which controls, when acquiring the positioninformation, a direction of a camera based on the position information.

A monitoring system according to the present invention includes a mobileterminal and a management device which communicates with the mobileterminal, in which the mobile terminal includes a first transmissionunit which transmits position information of the mobile terminal to themanagement device, and the management device includes a control unitwhich controls, when acquiring the position information, a direction ofa camera based on the position information, a tracking unit whichextracts one or more tracking target candidates based on an image/imagesacquired from the camera, and a display unit which displays theimage/images and highlights the tracking target candidates included inthe image/images so as to be selectable.

A mobile terminal according to the present invention includes a firsttransmission unit which transmits position information of the ownterminal to a management device at a predetermined interval, in whichthe first transmission unit starts transmitting the position informationto the management device when detecting a predetermined action.

A management device according to the present invention includes anacquisition unit which acquires position information of a mobileterminal at a predetermined interval and a control unit which controls,when acquiring the position information of the mobile terminal, adirection of a camera based on the position information.

Advantageous Effects of Invention

It is possible to appropriately control a camera when a position of auser of a mobile terminal changes or when a monitoring target isdifferent from the user of the mobile terminal.

BRIEF DESCRIPTION OF DRAWINGS

[FIG. 1] It depicts a block diagram showing an example of aconfiguration of a monitoring system according to a first exemplaryembodiment.

[FIG. 2] It depicts a flowchart showing information transmission of amobile terminal according to the first exemplary embodiment.

[FIG. 3] It depicts a flowchart showing camera control by a managementdevice according to the first exemplary embodiment.

[FIG. 4] It depicts a flowchart showing that the management deviceaccording to the first exemplary embodiment tracks a tracking target.

[FIG. 5] It depicts an overhead view showing a specific example of anaction when a security guard finds a suspicious person in the firstexemplary embodiment.

[FIG. 6] It depicts an overhead view showing a specific example of thecamera control by the management device according to the first exemplaryembodiment.

[FIG. 7] It depicts a diagram showing a specific example of an image ofa camera 300-1 in FIG. 6.

[FIG. 8] It depicts a diagram showing a specific example of an image ofa camera 300-2 in FIG. 6.

[FIG. 9] It depicts a diagram showing a specific example of an image ofa camera 300-3 in FIG. 6.

[FIG. 10] It depicts a diagram showing a specific example of a displayunit of the management device according to the first exemplaryembodiment.

[FIG. 11] It depicts a diagram showing a modification of the displayunit of the management device according to the first exemplaryembodiment.

[FIG. 12] It depicts a block diagram showing an example of aconfiguration of a monitoring system according to a second exemplaryembodiment.

[FIG. 13] It depicts a flowchart showing information transmission of amobile terminal according to the second exemplary embodiment.

[FIG. 14] It depicts a flowchart showing camera control by a managementdevice according to the second exemplary embodiment.

[FIG. 15] It depicts a flowchart showing that the management deviceaccording to the second exemplary embodiment tracks a tracking target.

[FIG. 16] It depicts an overhead view showing a specific example of thecamera control by the management device according to the secondexemplary embodiment.

[FIG. 17] It depicts a diagram showing a specific example of a displayunit of the management device according to the second exemplaryembodiment.

[FIG. 18] It depicts a flowchart showing information transmission of amobile terminal according to a third exemplary embodiment.

[FIG. 19] It depicts a flowchart showing that a management deviceaccording to the third exemplary embodiment tracks a tracking target.

[FIG. 20] It depicts a diagram showing a hardware configuration of themonitoring system according to the first exemplary embodiment.

DESCRIPTION OF EMBODIMENTS First Exemplary Embodiment (Feature of theInvention)

In a monitoring system according to the present exemplary embodiment, aposition where there is a mobile terminal is photographed based onposition information of the mobile terminal held by a security guard.Thus, a monitoring camera is controlled and tracking candidatesextracted from an image acquired by the monitoring camera are displayedto an operator of a management device. The feature is to prompt theoperator of the management device to perform selection from thedisplayed tracking candidates.

(Configuration)

FIG. 1 is a block diagram showing an example of a configuration of amonitoring system 1000 according to the present exemplary embodiment. Inthe present exemplary embodiment, the monitoring system 1000 is a systemfor monitoring facilities such as ports, airports, platforms, importantfacilities including power plants, plants, and dams, warehouses, leisurefacilities, stadiums, commercial facilities, buildings, cities, andstreets. More specifically, it will be described, as an exemplaryembodiment, that a facility in which monitoring by a security guard'spatrolling and cameras is performed is monitored. Referring to FIG. 1,the monitoring system 1000 includes a mobile terminal 1100, a managementdevice 1200, and cameras 300-1 to 300-N.

The mobile terminal 1100 includes a position acquisition unit 1110, amotion detection unit 1120, and a communication unit 1130. Examples ofthe mobile terminal 1100 include a wireless device, a mobile phone, asmartphone, and a tablet terminal. The position acquisition unit 1110,the motion detection unit 1120, and the like can also be implemented bycausing a computer constituting these devices to execute a computerprogram for performing processes using the hardware. As shown in FIG.20, a central processing unit (CPU) 100 provided in the mobile terminal1100 is connected to a memory 120 provided in the mobile terminal 1100,and functions based on a predetermined program read from the memory 120or various types of data, and whereby the position acquisition unit1110, the motion detection unit 1120, and the like may be implemented.More specific examples of the CPU 100 include a processor and amicroprocessor. The CPU 100 of the mobile terminal 1100 may be connectedto the communication unit 1130, an input accepting unit 110, and adisplay unit 130 via an input/output circuit. Here, the memory 120 maybe configured as a random access memory (RAM) or a read only memory(ROM).

The position acquisition unit 1110 is connected to the motion detectionunit 1120 and the communication unit 1130. The position acquisition unit1110 further acquires real world coordinates as position informationfrom a global positioning system (GPS) satellite. When acquiringposition information of the mobile terminal 1100, the positionacquisition unit 1110 transmits the position information of the mobileterminal 1100 to the communication unit 1130. In the present exemplaryembodiment, the position information is acquired using a GPS satellite,but it is also possible to use a system which transmits and receivesdata using wireless communication, and calculates position informationfrom the information of the access position. Here, the positioninformation is real world coordinates which are coordinates unified inthe real world, but is not necessarily limited thereto, and can containinformation such as current time. Here, the real world coordinates maybe constituted by information indicating longitude and latitude.

The motion detection unit 1120 is connected to the position acquisitionunit 1110 and the communication unit 1130. The motion detection unit1120 further detects a predetermined action of a security guard. Whendetecting the predetermined action of the security guard, the motiondetection unit 1120 transmits, to the communication unit 1130, controlswitching information for requesting mode switching of control of thecameras 300-1 to 300-N. Here, the predetermined action means that thesecurity guard operates the mobile terminal 1100. More specifically,when the security guard finds a suspicious person, a suspicious object,or the like during patrolling, the security guard operates the mobileterminal, and transmits, to the communication unit 1130, the controlswitching information requesting the mode switching of the control ofthe cameras 300-1 to 300-N. At this time, the security guard mayphotograph an image of a suspicious person, a suspicious object, or thelike with the camera (not shown) of the mobile terminal 1100 andtransmit the photographed image to the communication unit 1130 togetherwith the position information of the mobile terminal 1100. Here, modesof the camera control will be described later.

The communication unit 1130 includes a first communication unit 1131 anda second communication unit 1132. The communication unit 1130communicates with the management device 1200 via a communicationnetwork.

The first communication unit 1131 is connected to the positionacquisition unit 1110. The first communication unit 1131 transmits theposition information of the mobile terminal 1100 acquired by theposition acquisition unit 1110 to the management device 1200 via thecommunication network. The first communication unit 1131 can receiveinformation from the management device 1200. The first communicationunit 1131 may transmit an image of a suspicious person, a suspiciousobject, or the like photographed by the camera (not shown) of the mobileterminal 1100 to the management device 1200 together with the positioninformation of the mobile terminal 1100.

The second communication unit 1132 is connected to the motion detectionunit 1120. The second communication unit 1132 transmits the controlswitching information to the management device 1200 via thecommunication network when the motion detection unit 1120 detects thepredetermined action of the security guard. The second communicationunit 1132 can receive information from the management device 1200. Here,the first communication unit 1131 and the second communication unit 1132are only required to be functionally different and to independentlyperform transmission and reception. That is, the first communicationunit 1131 and the second communication unit 1132 are not necessarilyseparated and can be implemented by one as a hardware configuration. Thesecond communication unit 1132 may transmit the control switchinginformation together with an image of a suspicious person, a suspiciousobject, or the like photographed by the camera (not shown) of the mobileterminal 1100.

The management device 1200 includes a communication unit 1210, a controlunit 1220, and a display unit 1230. The management device 1200 isprovided in a building where there is a security company that monitorsan entire facility or in a security room installed in a monitoredfacility. The management device 1200 can be implemented when installedoutside the facility as a management server. Furthermore, the managementdevice 1200 can be not only an independent device but also beincorporated as one constituent element of the camera 300-1 to bedescribed later. Here, the control unit 1220 can be implemented by acomputer program causing a computer constituting these devices toperform processes by using the hardware. That is, as shown in FIG. 20, aCPU 200 provided in the management device 1200 is connected to a memory220 provided in the management device 1200, and functions based on apredetermined program or various types of data read from the memory 220,and whereby the control unit 1220 may be implemented. More specificexamples of the CPU 200 include a processor and a microprocessor. TheCPU 200 of the management device 1200 may be connected to thecommunication unit 1210, an input accepting unit 210, and the displayunit 1230 via an input/output circuit. A connection unit 230 of themanagement device 1200 is connected to the cameras 300-1 to 300-N. Here,the memory 220 may be configured as a RAM or a ROM.

The communication unit 1210 includes a first communication unit 1211 anda second communication unit 1212. The communication unit 1210 cantransmit and receive information to and from the communication unit 1130of the mobile terminal 1100. The communication unit 1210 furthertransmits the received information to the control unit 1220.

The first communication unit 1211 communicates with the firstcommunication unit 1131 of the mobile terminal 1100 and receives theabove position information of the mobile terminal 1100. The firstcommunication unit 1211 transmits the received position information ofthe mobile terminal 1100 to the control unit 1220. The firstcommunication unit 1211 can transmit information to the firstcommunication unit 1131. The first communication unit 1211 may acquirethe position information of the mobile terminal 1100 together with animage of a suspicious person, a suspicious object, or the likephotographed by the camera (not shown) of the mobile terminal 1100.

The second communication unit 1212 communicates with the secondcommunication unit 1132 of the mobile terminal 1100 via thecommunication network and receives the above control switchinginformation. The second communication unit 1212 transmits the controlswitching information to the control unit 1220. The second communicationunit 1212 can transmit information to the second communication unit1132. Here, the first communication unit 1211 and the secondcommunication unit 1212 are only required to be functionally differentand to independently perform transmission and reception. That is, thefirst communication unit 1211 and the second communication unit 1212 arenot necessarily separated and can be implemented by one as a hardwareconfiguration. The second communication unit 1212 may acquire thecontrol switching information together with an image of a suspiciousperson, a suspicious object, or the like photographed by the camera (notshown) of the mobile terminal 1100.

The control unit 1220 includes a tracking unit 1221 and a storage 1222.In addition, the control unit 1220 is connected to the communicationunit 1210, the display unit 1230, and the cameras 300-1 to 300-N. Thecontrol unit 1220 controls the cameras based on the control switchinginformation acquired from the communication unit 1210. Morespecifically, the control unit 1220 switches the camera control from anormal mode to a tracking mode in response to receiving the controlswitching information. Here, in the normal mode, a monitoring area isset to each of the cameras 300-1 to 300-N, and the control unit 1220controls each of the cameras 300-1 to 300-N to photograph the setmonitoring area. On the other hand, in the tracking mode, the controlunit 1220 controls the photographing range of each of the cameras 300-1to 300-N to the position indicated by the acquired position informationof the mobile terminal 1100 and tracks a tracking target. That is, thecontrol unit 1220 controls the Pan-Tilt-Zoom (PTZ) of the cameras 300-1to 300-N so that the cameras 300-1 to 300-N can photograph the positionindicated by the position information of the mobile terminal 1100. Thecontrol unit 1220 further receives the images photographed by thecameras 300-1 to 300-N and transmits them to the display unit 1230.Here, the tracking means that an object detected in an image frame whichis, for example, a previous image n-1 frame (n is a natural number) isto be determined as the identical object as an object detected in thecurrent image frame (n frame), and the movement state of the objectregarded to be identical is measured. More specifically, it means thatthe pan, tilt, and zoom of the camera is controlled so that the trackingtarget object is included in the image obtained by, for example, thecamera 300-1.

When the mode is changed to the tracking mode, the tracking unit 1221selects tracking candidates based on the images photographed by thecameras 300-1 to 300-N. The tracking unit 1221 starts tracking thetracking target designated by the operator of the management device 1200from the tracking candidates. The detailed tracking procedure will bedescribed later.

The storage 1222 is a storage medium such as a hard disc drive (HDD).The storage 1222 stores the images photographed by the cameras 300-1 to300-N, the information on the positions where the cameras 300-1 to 300-Nare installed, the moving direction of the mobile terminal 1100, theacceleration direction of the mobile terminal 1100, the controlswitching information, the position information of the mobile terminal1100, the feature amounts of specific objects (people, animals, smallairplanes, helicopters, and the like), the feature amounts of individualpeople, clothes, and the like of security guards.

The display unit 1230 includes a liquid crystal display device. Thedisplay unit 1230 displays the camera image obtained by the control unit1220. The display unit 1230 further displays information on trackingtarget candidates, which is candidates for a tracking target, anddisplays information for prompting the operator to select a trackingtarget from the tracking candidates.

Each of the cameras 300-1 to 300-N is a complementary metal oxidesemiconductor (CMOS) camera. Each of the cameras 300-1 to 300-N canchange the photographing range by the Pan-Tilt-Zoom (PTZ) control. Here,in the present exemplary embodiment, the camera is a CMOS camera, butmay be a charge coupled devices (CCD) camera, an infrared camera, or thelike, and may be any camera as long as it is image acquisition meanswhich can acquire an image from which a feature amount can be extracted.Furthermore, the cameras 300-1 to 300-N are not necessarily the samecamera, and may be various cameras. In the present exemplary embodiment,it has been exemplified that the camera 300-1 and the management device1200 are configured as separate devices, but the monitoring system 1000may include an intelligent camera in which the camera 300-1 and themanagement device 1200 are integrated. Furthermore, the monitoringsystem 1000 may have a configuration in which the management device 1200is embedded in each of the cameras 300-1 to 300-N, or the managementdevice 1200 is incorporated only in the camera 300-1 and controls theother cameras.

(Control by Mobile Terminal)

Next, a control procedure in the mobile terminal 1100 will be describedwith reference to FIG. 2. FIG. 2 is a flowchart showing that the mobileterminal 1100 transmits information to the management device 1200.

In step S110, the position acquisition unit 1110 acquires the positioninformation of the own terminal from the GPS satellite. The positionacquisition unit 1110 transmits the acquired position information to thefirst communication unit 1131. Here, the position acquisition unit 1110acquires the position information at a predetermined interval (forexample, every 10 seconds). The predetermined interval may be a variablecycle or a random cycle. As another example of the predeterminedinterval, the position acquisition unit 1110 may acquire the positioninformation when a request from the security guard to the mobileterminal 1100 is made. In this case, the predetermined interval is thetime when a request from the security guard to the mobile terminal 1100is made.

In step S120, the motion detection unit 1120 accepts, from the securityguard, control switching information which is a request for switchingmonitoring control (more specifically, camera control) from the normalmode to the tracking mode. The motion detection unit 1120 transmits theacquired control switching information to the second communication unit1132. Here, sequentially in steps S110 and S120, the positionacquisition unit 1110 acquires the position information of the mobileterminal 1100 separately and independently from the process that themotion detection unit 1120 acquires the control switching information.The position acquisition unit 1110 and the motion detection unit 1120may acquire each information at the same time. The motion detection unit1120 may acquire the control switching information together with animage of a suspicious person, a suspicious object, or the likephotographed by the camera (not shown) of the mobile terminal 1100.

In step S130, the first communication unit 1131 transmits the acquiredposition information of the mobile terminal 1100 to the managementdevice 1200 via the first communication unit 1211. The secondcommunication unit 1132 transmits the acquired control switchinginformation to the management device 1200 via the second communicationunit 1212. Here, the first communication unit 1131 transmits informationto the management device 1200 every time the position acquisition unit1110 acquires the position information of the mobile terminal 1100 at apredetermined interval (for example, periodically). That is, the firstcommunication unit 1131 transmits the position information of the mobileterminal 1100 to the management device 1200 at the predeterminedinterval (for example, periodically). Alternatively, the firstcommunication unit 1131 or the second communication unit 1132 maytransmit an image of a suspicious person, a suspicious object, or thelike photographed by the camera (not shown) of the mobile terminal 1100.Here, the predetermined interval for acquiring the position informationby the position acquisition unit 1110 and the predetermined interval fortransmitting the position information by the first communication unit1131 may be synchronous or asynchronous.

(Camera Control by Management Device)

Next, a procedure in which the management device 1200 controls thecameras 300-1 to 300-N will be described with reference to FIG. 3. FIG.3 is a control flowchart showing that the management device 1200controls the photographing ranges of the cameras 300-1 to 300-N.

In step S210, the first communication unit 1211 receives the positioninformation of the mobile terminal 1100 from the mobile terminal 1100and transmits the acquired information to the control unit 1220. Thesecond communication unit 1212 receives the control switchinginformation from the mobile terminal 1100, and transmits the acquiredinformation to the control unit 1220. Alternatively, the firstcommunication unit 1211 or the second communication unit 1212 mayreceive an image of a suspicious person, a suspicious object, or thelike photographed by the camera (not shown) of the mobile terminal 1100.

In step S220, the control unit 1220 switches the mode of camera controlfrom the normal mode to the tracking mode based on the control switchinginformation. The control unit 1220 further controls the pan, tilt, andzoom of each of the cameras 300-1 to 300-N based on the acquiredposition information of the mobile terminal 1100. The more detailedalgorithm is described below. First, the storage 1222 stores thepositions which the cameras can photograph in advance. The control unit1220 extracts combinations in which the real world coordinates which arethe acquired position information are within the photographable range ofthe combinations of pan, tilt, and zoom stored in the storage 1222.Then, the control unit 1220 controls the pan, tilt, and zoom of thecamera 300-1 to a target value which is the combination closest to thecurrent combination among the extracted combinations of pan, tilt, andzoom. Here, the control unit 1220 selects the combination closest to thecurrent combination, but may select a combination in which the realworld coordinates which are the acquired position information are at thecenter of the camera.

The control unit 1220 can further detect the moving direction of thesecurity guard and control the camera by predicting the current positionof the security guard. Specifically, the control unit 1220 periodicallyreceives the position information and the transmission time informationof the mobile terminal 1100. Thereafter, the control unit 1220calculates the moving direction and the moving speed of the securityguard from the position information and transmission time of the mobileterminal 1100 received this time, and the position information andtransmission time of the mobile terminal 1100 previously received. Next,the control unit 1220 calculates the expected moving amount based on thedifference between the time at which the current position information isreceived and the current time, and the moving speed. The control unit1220 predicts the position where the mobile terminal 1100 is to belocated at the current time based on the calculated moving direction andexpected moving amount of the security guard. Finally, the control unit1220 controls the pan, tilt, and zoom of each of the cameras 300-1 to300-N by the above method so as to include the predicted positioninformation in the photographing range.

The control unit 1220 can further detect the moving direction of thesecurity guard and control the cameras by predicting the future positionof the security guard. Specifically, the control unit 1220 predicts thefuture position by performing the process similar to the above processfor predicting the current position of the security guard to the futuretime as the prediction of the future position of the security guard.

In step S230, the control unit 1220 transmits the control amountcalculated in step S220 to the cameras 300-1 to 300-N. After that, theprocedure is ended. Here, the control unit 1220 may sequentially updatethe camera positions of the cameras 300-1 to 300-N based on theperiodically acquired position information of the mobile terminal. Withthis configuration, if the security guard holding the mobile terminal1100 has moved away from the previously acquired position, thephotographing ranges of the cameras are updated, and it is possible toincrease the possibility that the cameras 300-1 to 300-N capture thesecurity guard.

(Tracking Control by Management Device)

Next, a procedure of tracking control by the management device 1200 willbe described with reference to FIG. 4. FIG. 4 is a flowchart of trackingcontrol in which the management device 1200 performs tracking.

In step S310, when acquiring a mode change request, the control unit1220 changes the camera control from the normal mode to the trackingmode.

In step S320, the tracking unit 1221 acquires the images photographed bythe cameras 300-1 to 300-N. The tracking unit 1221 stores the imagesacquired from the cameras 300-1 to 300-N in the storage 1222.

In step S330, the tracking unit 1221 extracts the feature amount fromthe images acquired from the cameras 300-1 to 300-N. The tracking unit1221 compares the extracted feature amount with the feature amount whichis stored in advance in the storage 1222 and specifies individualpeople, and extracts person candidates. The tracking unit 1221 selectsthe tracking candidates from the extracted person candidates. Here, thetracking unit 1221 may acquire motion vectors indicating the movingamounts and the moving directions of a moving objects, and select onlythe moving objects as the tracking candidates. More specifically, thetracking unit 1221 reads a plurality of frames and compares the imagedata between flaims, and acquires the motion vector. Alternatively, thetracking unit 1221 may extract the tracking candidates by extracting afeature amount from the image of the suspicious person photographed bythe camera (not shown) of the mobile terminal 1100 acquired by the firstcommunication unit 1211 or the second communication unit 1212, andcomparing the extracted feature amount with the feature amount extractedfrom the images acquired from the cameras 300-1 to 300-N. In thetracking control by the management device 1200, a person is extracted asa tracking candidate, but the tracking candidate is not limited to aperson, and may be an animal, a moving object (a car or a motorcycle),or an aircraft (an airplane or a helicopter).

In step S340, the tracking unit 1221 outputs the images photographed bythe cameras 300-1 to 300-N in step S320 and the tracking candidatesextracted in step S330 to the display unit 1230. The display unit 1230displays the images photographed by the cameras 300-1 to 300-N andhighlights the tracking candidates extracted in step S330 in the images.

In step S350, the display unit 1230 accepts designation of one or moretracking targets by the operator. Here, the display unit 1230 acceptsthe designation by the operator, but the designation is not particularlylimited, and the control unit 1220 may accept the designation by theoperator.

In step S360, the tracking unit 1221 tracks the tracking target based onthe designation of the tracking target. The specific detection algorithmof the tracking target is described below. The tracking unit 1221predicts the position of the tracking target in the current frame basedon the position of the tracking target detected in the previous frame.As this method, various existing methods such as a method using a Kalmanfilter or a particle filter can be used. Then, the tracking unit 1221performs tracking based on the predicted position of the tracking targetin the current frame and the image in the current frame. Note that, thetracking is not particularly limited to the above exemplary embodiment,and may be performed by associating the similarity or the likelihood ofthe feature amount of the tracking target with that of the object in thecurrent frame. In addition, the control unit 1220 may transmit theimages photographed by the cameras 300-1 to 300-N to the mobile terminal1100 via the communication unit 1210. With the above configuration, ifthe security guard loses sight of the suspicious person, tracking can bestarted again. Alternatively, when there are a plurality of securityguards and a plurality of mobile terminals, the control unit 1220 maytransmit the image to the mobile terminals other than the mobileterminal the position information of which is acquired. Furthermore,while the tracking unit 1221 is tracking the tracking target, thecontrol unit 1220 may analyze images photographed by the cameras 300-1to 300-N generate metadata related to the tracking target, and store themetadata in the storage 1222 in association with the images. Here, themetadata is data of clothe colors, features of faces, and the like ofthe tracking target.

(Specific Example of First Exemplary Embodiment)

Next, a specific example of the first exemplary embodiment will bedescribed with reference to FIGS. 5 to 10. FIG. 5 is an overhead viewshowing a specific example of an action when the security guard finds asuspicious person in the first exemplary embodiment.

As a specific example, there are a building A, a building B, and abuilding C in a certain facility. The building A is provided with thecamera 300-1 and the camera 300-2. The building B is provided with thecamera 300-3. The suspicious person is at the center of FIG. 5. Thesecurity guard is at the lower right of FIG. 5. When the security guardfinds the suspicious person, the security guard performs, to the mobileterminal 1100, an input operation to request the mode change. Whenaccepting the input, the mobile terminal 1100 transmits the controlswitching information to the management device 1200 together with theposition information of the mobile terminal 1100. Then, the securityguard starts moving in the direction toward the suspicious person tosecure the suspicious person. While the suspicious person is moving, themobile terminal 1100 sequentially transmits the acquired positioninformation to the management device 1200.

FIG. 6 is an overhead view of camera control by the management deviceaccording to the first exemplary embodiment. In FIG. 6, when acquiringthe control switching information from the mobile terminal 1100, themanagement device 1200 switches the mode for controlling the cameras300-1 to 300-3 to the tracking mode. The management device 1200 turns,based on the acquired position information of the mobile terminal 1100,the visual-line directions of the cameras 300-1 to 300-3 to the acquiredposition of the mobile terminal 1100 (where the security guard isexpected to exist).

FIG. 7 is a diagram showing an image example photographed by the camera300-1. The security guard and the suspicious person are photographedfrom a distance by the camera 300-1. FIG. 8 is a diagram showing animage example photographed by the camera 300-2. The security guard andthe suspicious person are photographed from a medium distance by thecamera 300-2. FIG. 9 is a diagram showing an image example photographedby the camera 300-3. The security guard and only the upper body of thesuspicious person are photographed from a short distance by the camera300-3. Since the control unit 1220 controls the focus target of thecamera 300-3 to the position of the mobile terminal 1100, the camera300-3 cannot capture the suspicious person. The images photographed bythe cameras 300-1 to 300-3 are transmitted to the management device 1200and stored in the storage 1222.

FIG. 10 is a diagram showing a specific example of the display unit ofthe management device according to the first exemplary embodiment. Thecontrol unit 1220 displays the images photographed by the cameras 300-1to 300-3 on the display unit 1230. The control unit 1220 furtherextracts a tracking candidate 1 to a tracking candidate 5 appearing ineach camera image. The control unit 1220 highlights the trackingcandidate 1 to the tracking candidate 5 by encircling them in the imagesphotographed by the cameras 300-1 to 300-3. Furthermore, one or moretracking candidates can be selected so that the tracking target can bedetermined from the tracking candidate 1 to the tracking candidate 5. Inthis specific example, the operator of the management device selects thetracking candidate 1 and the tracking candidate 3 from the trackingcandidate 1 to the tracking candidate 5 being displayed and presses thedetermination button, whereby the tracking candidate 1 and the trackingcandidate 3 are determined as the tracking targets. The tracking unit1221 determines the determined tracking targets as tracking targets, andtracks the determined tracking targets. More specifically, the controlunit 1220 controls the camera 300-1 so that the tracking target 1 isphotographed around the center of the screen. The control unit 1220further controls the camera 300-2 so that the tracking target 3 isphotographed around the center of the screen.

FIG. 11 shows a modification. In FIG. 10, the operator needs to executesteps of selecting and determining, and the two operations are required.Here, as shown in FIG. 11, the tracking candidates highlighted in theimages acquired by the cameras 300-1 to 300-3 may be directly selected.More specifically, the “tracking target 1” in FIG. 11 has turned gray,which indicates a state in which it is selected as a tracking target.The “tracking object 2” has turned white, which indicates a state inwhich it is not selected as a target. When the “tracking target 1” inFIG. 11 is pressed by the operator, it turns gray as shown in FIG. 11,and the tracking is started. When the “tracking target 1” in FIG. 11 ispressed by the operator again, is turns white, and the tracking isstopped. With the above configuration, it is possible to improvevisibility and operability of the operator.

(Function and Effect)

Since the position information of the mobile terminal can besequentially acquired, it is possible to increase the possibility thatthe camera can capture the security guard or the suspicious person ifthe security guard is moving. Furthermore, it is possible to select atracking target which is not a security guard who is a user of themobile terminal by controlling the cameras and displaying trackingcandidates so as to be selectable based on the position information ofthe mobile terminal. Moreover, the mobile terminal transmits the imageof a tracking target such as a suspicious person photographed by itselfto the management device in accordance with the transmission of theposition information, and whereby it is possible to increase thelikelihood of extraction of tracking candidates by the management deviceand to narrow down tracking candidates.

Second Exemplary Embodiment

In a monitoring system according to the present exemplary embodiment,presence or absence of tracking by a security guard is estimated usingan acceleration sensor of a mobile terminal, and when it is determinedthat the security guard performs tracking, the mobile terminal transmitscontrol switching information to a management device. Furthermore, onemonitoring camera to be controlled is selected from a plurality ofmonitoring cameras. Moreover, before tracking candidates are extractedand an operator of the management device selects a tracking target,tracking a security guard as a provisional tracking target is started.

(Configuration)

FIG. 12 is a block diagram showing an example of a configuration of amonitoring system 2000 according to the present exemplary embodiment.Referring to FIG. 12, the monitoring system 2000 includes a mobileterminal 2100, a management device (management server) 2200, and cameras300-1 to 300-N.

The mobile terminal 2100 includes at least a position acquisition unit1110, a motion detection unit 2120, a communication unit 1130, and anacceleration sensor 2140. The mobile terminal 2100 has all the hardwareconfiguration of the mobile terminal 1100 in the first exemplaryembodiment shown in FIG. 20, and further includes the accelerationsensor 2140.

The position acquisition unit 1110 and the communication unit 1130 aresimilar to those in the first exemplary embodiment.

The motion detection unit 2120 is connected to the position acquisitionunit 1110, the communication unit 1130, and the acceleration sensor2140. The motion detection unit 2120 detects a predetermined action of asecurity guard based on the output value of the acceleration sensor2140. When the predetermined action of the security guard is detected,the motion detection unit 2120 transmits, to the communication unit1130, control switching information for requesting mode change of cameracontrol to a tracking mode. The motion detection unit 2120 furthertransmits control the switching information to the position acquisitionunit 1110. Here, the predetermined action means that the security guardhas found a suspicious person or a suspicious object and is in atracking state. The motion detection unit 2120 may independentlytransmit the control switching information by the operation of thesecurity guard similarly to the first exemplary embodiment. Theacceleration sensor 2140 may be expressed as a motion detection devicewhich detects vertical acceleration and horizontal acceleration of themobile terminal 2100.

The management device 2200 includes a communication unit 1210, a controlunit 2220, and a display unit 1230. The management device 2200 has asimilar hardware configuration to the management device 1200 accordingto the first exemplary embodiment shown in FIG. 20.

The communication unit 1210 and the display unit 1230 are similar tothose in the first exemplary embodiment.

The control unit 2220 includes a tracking unit 2221, a storage 1222, anda selection unit 2223. The control unit 2220 performs camera controlsimilarly to the control unit 1220 in the first exemplary embodiment.The control unit 2220 controls the cameras based on the controlswitching information acquired by the communication unit 1210. Morespecifically, the camera is controlled by changing the mode of thecamera control from a normal mode to a tracking mode similarly to thefirst exemplary embodiment.

When the mode is changed to the tracking mode, the tracking unit 2221extracts a provisional tracking target as a tentative tracking targetfrom the images photographed by the cameras 300-1 to 300-N. In addition,when the mode is changed to the tracking mode, the tracking unit 2221extracts tracking candidates from the images photographed by the cameras300-1 to 300-N. The tracking unit 2221 further tracks the trackingtarget specified by the operator of the management device 2200 from theextracted tracking candidates.

The storage 1222 is similar to that in the first exemplary embodiment.

The selection unit 2223 selects the camera closest to the position ofthe mobile terminal 2100 based on the acquired position information ofthe mobile terminal 2100 and the position information of each of thecameras 300-1 to 300-N stored in advance in the storage 1222. Here, theselection unit 2223 selects the closest camera, but the camera is notparticularly limited to the closest camera. Specifically, the selectionunit 2223 may select the farthest camera. The selection unit 2223 doesnot necessarily select one camera, and may select all the cameras withina predetermined range from the mobile terminal 1100.

The cameras 300-1 to 300-N are similar to those in the first exemplaryembodiment.

(Control by Mobile Terminal)

Next, a control procedure in the mobile terminal 2100 will be describedwith reference to FIG. 13. FIG. 13 is a flowchart showing that themobile terminal 1100 transmits information to the management device2200.

In step S410, a similar process to step S110 in FIG. 2 is performed.

In step S420, the motion detection unit 2120 accepts the acceleration ofthe mobile terminal 2100 from the acceleration sensor 2140.

In step S430, the motion detection unit 2120 determines whether theacquired acceleration is equal to or greater than a predetermined value.When the acceleration is equal to or greater than the predeterminedvalue, the motion detection unit 2120 determines that the security guardhas performed the predetermined action. The motion detection unit 2120transmits the control switching information to the second communicationunit 1132, and the process proceeds to step S440. When the accelerationis less than the predetermined value, the motion detection unit 2120determines that the security guard has not performed the predeterminedaction, and the process returns to step S420 to detect the accelerationagain. Here, the motion detection unit 2120 determines that the securityguard has performed the predetermined action simply by checking thepredetermined value of the acceleration, but the motion detection unit2120 may determine that the security guard has performed thepredetermined action when the period of time in which the accelerationis equal to or greater than the predetermined value exceeds a certainperiod.

In step S440, a similar process to step S130 in FIG. 2 is performed, andthe procedure is ended.

(Camera Control by Management Device)

Next, a procedure in which the management device 2200 controls thecameras 300-1 to 300-N will be described with reference to FIG. 14 andthe following Table 1. FIG. 14 is a control flowchart showing that themanagement device 2200 controls the photographing ranges of the cameras300-1 to 300-N. Table 1 shows the position information of each of thecameras 300-1 to 300-N stored in the storage 1222.

In step S510, a similar process to step S210 in FIG. 3 is performed.

In step S520, the selection unit 2223 compares the acquired positioninformation of the mobile terminal 2100 with the position information ofeach of the cameras 300-1 to 300-N as shown in the following Table 1,and selects the camera closest to the position of the mobile terminal2100. More specifically, when the real world coordinates, which are theacquired position information of the mobile terminal 2100, are at 35degrees 63 minutes 81.100 seconds north latitude and at 139 degrees 74minutes 72.894 seconds east longitude, the selection unit 2223 comparesthe degrees of north latitude and east longitude of each camera with thedegrees of north latitude and east longitude of the acquired positioninformation of the mobile terminal 2100, and calculates the difference.The selection unit 2223 selects the camera having the smallestcalculated difference from the cameras 300-1 to 300-N. When a pluralityof cameras is selected, the selection unit 2223 calculates eachdifference using the minutes of north latitude and east longitude, andselects the camera having the smallest difference. When a plurality ofcameras is further selected, the selection unit 2223 performs comparisonwith the seconds of north latitude and east latitude, and selects thecamera having the smallest difference. At this point, when there is aplurality of cameras having the smallest difference, the selection unit2223 selects all of the cameras as target cameras. In this specificexample, the camera 300-N is selected.

TABLE 1 Camera number Camera position information 300-1 lat. 35°64′95.298″ N. long. 139° 74′84.257″ E. 300-2 lat. 35° 65′13.553″ N.long. 139° 74′91.684″ E. . . . . . . lat. 35° 64′83.119″ N. 300-N long.139° 74′71.894″ E.

In step S530, a similar process to step S220 in FIG. 3 is performed.Here, in this step, it is possible to further detect the movingdirection of the security guard and control the camera by predicting thecurrent position of the security guard. Specifically, the control unit2220 periodically receives the position information and the transmissiontime information of the mobile terminal 2100. In addition, the controlunit 2220 receives the acceleration and the acceleration direction ofthe mobile terminal 2100 acquired by the acceleration sensor 2140 of themobile terminal 2100. Thereafter, the control unit 2220 calculates theexpected moving amount of the security guard from the positioninformation, transmission time information, and acceleration of themobile terminal 2100 received this time. Next, the control unit 2220predicts the position where the mobile terminal 2100 is to be located atthe current time based on the calculated expected moving amount and theacquired acceleration direction. Finally, the control unit 2220 controlsthe pan, tilt, and zoom of each of the cameras 300-1 to 300-N by theabove method so as to include the predicted position of the mobileterminal 2100 in the photographing range.

In step S540, a similar process to step S230 in FIG. 3 is performed, andthe procedure is ended.

(Tracking Control by Management Device)

Next, a procedure of tracking control by the management device 2200 willbe described with reference to FIG. 15. FIG. 15 is a flowchart oftracking control in which the management device 2200 performs tracking.

In step S610, a similar process to step S310 in FIG. 4 is performed.

In step S620, a similar process to step S320 in FIG. 4 is performed.

In step S630, the tracking unit 2221 extracts a feature amount from theacquired image. The tracking unit 2221 compares the acquired featureamount with the feature amount of security guards stored in advance inthe storage 1222, and extracts a security guard. Then, the tracking unit2221 determines the extracted security guard as a provisional trackingtarget. The tracking unit 2221 further tracks the determined provisionaltracking target. The tracking method is similar to the trackingalgorithm in the first exemplary embodiment.

In step S640, a similar process to step S330 in FIG. 4 is performed.

In step S650, the tracking unit 2221 outputs, to the display unit 1230,the images photographed by the cameras 300-1 to 300-N acquired in stepS620, the provisional tracking target set in step S630, and the trackingcandidates extracted in step S640. The display unit 1230 displays theimages photographed by the cameras 300-1 to 300-N acquired in step S620and highlights the provisional tracking target set in step S630 and thetracking candidates extracted in step S640 by encircling them in thescreen. As the highlighting, various method of enclosing a detectionobject by a line, filling a detection object with a marker, or the likecan be used.

In step S660, a similar process to step S350 in FIG. 4 is performed.

In step S670, the tracking unit 2221 changes the tracking target fromthe provisional tracking target to the designated tracking target basedon the designation of the tracking target, and continues the tracking.The specific detection algorithm of the tracking target is similar tothat in the first exemplary embodiment.

(Specific Example of Second Exemplary Embodiment)

Next, a specific example of the second exemplary embodiment will bedescribed with reference to FIGS. 5, 16, and 17. FIG. 5 shows aconfiguration similar to that described in the first exemplaryembodiment. In FIG. 5, when the security guard finds a suspiciousperson, the security guard moves in the direction toward the suspiciousperson to secure the suspicious person without operating the mobileterminal 2100. The acceleration sensor 2140 of the mobile terminal 2100acquires the acceleration of the security guard. When detecting that theacquired acceleration is equal to or greater than the predeterminedacceleration, the motion detection unit 2120 transmits control switchinginformation to the management device 2200 via the second communicationunit 1132.

FIG. 16 is an overhead view of camera control by the management deviceaccording to the second exemplary embodiment. In FIG. 16, the controlunit 2220 acquires the images from the cameras 300-1 to 300-3. Thecontrol unit 2220 selects the camera 300-2 closest to the position ofthe mobile terminal 2100 based on the position information of each ofthe cameras 300-1 to 300-3 and the position information of the mobileterminal 2100. The control unit 2220 further turns the visual-linedirection of the camera 300-2 to the acquired position of the mobileterminal 2100.

FIG. 17 is a diagram showing a specific example of the display unit ofthe management device according to the second exemplary embodiment. Thecontrol unit 2220 displays the images photographed by the cameras 300-1to 300-3 on the display unit 1230. The control unit 2220 extracts thesecurity guard appearing in the camera images and determines it as theprovisional tracking target. The control unit 2220 further extracts thetracking candidate 1 appearing in the camera images. The display unit1230 displays the images photographed by the cameras 300-1 to 300-3 andhighlights the provisional tracking target and the tracking candidate 1by encircling them. The control unit 2220 further controls the displayunit 1230 to display one or more tracking candidates so as to beselectable so that the tracking target can be determined from thetracking candidates. That is, with respect to the tracking candidate 1being displayed on the display unit 1230, the operator of the managementdevice selects the tracking candidate 1 and presses the determinationbutton, and whereby the tracking candidate 1 is changed to the trackingtarget this time. The control unit 2220 further controls the cameras totrack the determined tracking target.

(Function and Effect)

Since the mobile terminal transmits the control switching information tothe management device when the presence or absence of the tracking bythe security guard is estimated using the acceleration sensor of themobile terminal and the presence of the tracking by the security guardis estimated, it is possible to notify the management device of theinformation for changing the mode when the security guard cannotdirectly operate the mobile terminal nor transmit the control switchinginformation, that is, when the security guard needs to immediately tracka suspicious person or a suspicious object. Furthermore, by selecting amonitoring camera to be controlled from a plurality of monitoringcameras, it is possible to capture an object with a minimum number ofcameras by restricting the number of monitoring cameras. Moreover, bystarting tracking the security guard as a provisional tracking targetbefore tracking candidates are extracted and the operator of themanagement device selects a tracking target, it is possible to reducethe communication amount and the communication time with the mobileterminal.

Third Exemplary Embodiment (Feature of the Invention)

In a configuration in which a mobile terminal periodically transmitsposition information to a management device, a cycle of acquiring theposition of the mobile terminal is shorten when control switchinginformation is transmitted. When the operator of the management devicedetermines a tracking target, the mobile terminal ends the control forshortening the period. In addition, a provisional tracking target isautomatically switched to a tracking target which is the object to beactually tracked.

(Configuration)

The functional block and the hardware configuration in the thirdexemplary embodiment are similar to those in the second exemplaryembodiment.

(Control by Mobile Terminal)

Next, a control procedure in a mobile terminal 2100 will be describedwith reference to FIG. 18. FIG. 18 is a flowchart showing that themobile terminal 2100 transmits information to a management device 2200.

In step S710, a similar process to step S110 in FIG. 2 is performed.

In step S720, a similar process to step S120 in FIG. 2 is performed.

In step S730, a position acquisition unit 1110 acquires controlswitching information from a motion detection unit 2120. When acquiringthe control switching information, the position acquisition unit 1110changes the acquisition cycle of the position information to be acquiredby itself to a high-speed cycle faster than a low-speed cycle which is anormal acquisition cycle. Specifically, when accepting the controlswitching information during updating is performed with a cycle of tenseconds (low-speed cycle) in the case where there is no controlswitching information, the position acquisition unit 1110 changes theposition acquiring cycle of itself to a cycle of two seconds (high-speedcycle). Here, a first communication unit 1131 transmits the positioninformation to the management device 2200 every time the positioninformation acquired by the position acquisition unit 1110 is acquired,but a cycle may be set in the first communication unit 1131 and bechanged according to the control switching information.

In step S740, a similar process to step S130 in FIG. 2 is executed.

In step S750, the position acquisition unit 1110 determines whether atracking unit 2221 of the control unit 2220 has acquired informationindicating that the tracking target has been set. When the positionacquisition unit 1110 determines that the tracking unit 2221 has notacquired the information indicating that the tracking target has beenset, the process remains at step S750. When the position acquisitionunit 1110 determines that the tracking unit 2221 has acquired theinformation indicating that the tracking target has been set, theprocess proceeds to step S760.

In step S760, when the tracking unit 2221 has acquired the informationindicating that the tracking target has been set, the positionacquisition unit 1110 changes the cycle from the high-speed cycle to thelow-speed cycle. After that, the procedure is ended.

(Camera Control by Management Device)

Since the camera control by the management device is a similar controlprocedure to that in FIG. 14 in the second exemplary embodiment, thedescription thereof is omitted.

(Tracking Control by Management Device)

Next, a control procedure of tracking control in the management device2200 will be described with reference to FIG. 19. FIG. 19 is a flowchartof tracking control in which the management device 2200 performstracking.

In step S810, a similar process to step S310 in FIG. 4 is performed.

In step S820, a similar process to step S320 in FIG. 4 is performed.

In step S830, a similar process to step S630 in FIG. 15 is performed.

In step S840, the control unit 2220 transmits information indicatingthat the provisional tracking target has been set to the mobile terminal2100 via the communication unit 1210. Here, the control unit 2220transmits to the mobile terminal 2100 the information indicating thatthe provisional tracking target has been set, but may transmit to themobile terminal 2100 the information indicating that the tracking targethas been set when the tracking target is determined in step S870 whichwill be described later.

In step S850, the tracking unit 2221 detects, from the imagesphotographed by cameras 300-1 to 300-N, a moving object the comparisonresult of which with the provisional tracking target satisfies apredetermined condition. Then, the tracking unit 2221 sets the detectedmoving object as a new tracking target. That is, the tracking unit 2221changes the tracking target from the provisional tracking target whichis the security guard to the detected moving object. A more specificdetermination algorithm will be described below. First, the trackingunit 2221 detects the provisional tracking target and at least one othermoving object from a plurality of frames in the images photographed bythe cameras 300-1 to 300-N. The tracking unit 2221 compares the imagedata of the read frames, and acquires the motion vector indicating themoving amount and the moving direction of the provisional trackingtarget in the image (the angle of view), and the motion vectorindicating the moving amount and the moving direction of the othermoving object. Next, the tracking unit 2221 sets, as a new trackingtarget, the other moving object having the motion vector of thesubstantially same direction as that of the provisional tracking targetand the difference of the moving amounts is less than or equal to apredetermined value. At this time, when there are moving objects but arenot other moving objects satisfying the predetermined condition, or whena plurality of other moving objects satisfying the predeterminedcondition is detected, the tracking unit 2221 highlights the extractedmoving objects on the display unit 1230 as tracking candidates. Thecontrol unit 2220 requests the operator of the management device 2200 toselect a tracking target.

In step S860, the tracking unit 2221 sets the tracking target set instep S850 as the new tracking target, and continues tracking. Afterthat, the procedure is ended. The specific detection algorithm of thetracking target is similar to that in the first exemplary embodiment.

(Function and Effect)

By shortening the cycle of acquiring the position of the mobile terminaluntil the tracking target is determined, it is possible to improve thereal-time property of the position information which can be transmittedto the management device, and by setting the trigger at the end of thecontrol for shortening the cycle as the tracking target, it is possibleto prevent the cycle of acquiring the position of the mobile terminalfrom being excessively lengthened. In addition, since the provisionaltracking target is automatically switched to the tracking target whichis the object to be actually tracked, it is possible to quickly capturethe tracking target.

As described above, the present invention has been applied to the aboveexemplary embodiments as examples. However, the technical scope of thepresent invention is not limited to the above-described exemplaryembodiments. It is apparent for those skilled in the art to variouslymodify or improve such exemplary embodiments. In such a case, newexemplary embodiments with such modifications or improvements can beincluded in the technical scope of the present invention. This isapparent from the matters described in the claims.

[Supplementary Note 1]

A monitoring system including:

a mobile terminal; and

a management device which communicates with the mobile terminal, inwhich

the mobile terminal includes a first transmission unit which transmitsposition information of the mobile terminal to the management device ata predetermined interval, and

the management device includes a control unit which controls, whenacquiring the position information, a direction of a camera based on theposition information.

[Supplementary Note 2]

The monitoring system according to supplementary note 1, in which thecontrol unit controls the direction of the camera to a photographingrange in which a position indicated by the position information isphotographed.

[Supplementary Note 3]

The monitoring system according to supplementary note 1, in which

the mobile terminal includes a second transmission unit which transmits,to the management device, control switching information for switchingcontrol of the camera when detecting a predetermined action, and

the control unit controls, when receiving the control switchinginformation, the direction of the camera based on the positioninformation of the mobile terminal.

[Supplementary Note 4]

The monitoring system according to a supplementary note 1 or 2, in which

the management device has a normal mode for monitoring a predeterminedrange and a tracking mode for performing monitoring based on theposition information, and

the control unit switches, when receiving the control switchinginformation, the normal mode to the tracking mode and controls thedirection of the camera based on the position information of the mobileterminal.

[Supplementary Note 5]

The monitoring system according to any one of supplementary notes 1 to4, in which

the control unit acquires position information of each camera, and

the control unit selects at least one camera to be controlled from thecamera based on the acquired position information of the camera and theacquired position information of the mobile terminal.

[Supplementary Note 6]

The monitoring system according to any one of supplementary notes 1 to5, in which

the management device includes:

a tracking unit which extracts one or more tracking target candidatesbased on an image/images acquired from the camera; and

a display unit which displays the image/images and highlights thetracking target candidates included in the image/images so as to beselectable.

[Supplementary Note 7]

The monitoring system according to supplementary note 6, in which thetracking unit determines the selected candidate as a tracking target andtracks the tracking target.

[Supplementary Note 8]

The monitoring device according to supplementary note 6 or 7, in whichthe tracking unit identifies a holder of the mobile terminal based onthe image/images acquired from the camera, determines the identifiedholder as a provisional tracking target, and tracks the provisionaltracking target.

[Supplementary Note 9]

The monitoring device according to any one of supplementary notes 6 to8, in which

the tracking unit further extracts a moving object different from theholder of the mobile terminal based on the image/images acquired fromthe camera, and

the display unit highlights the extracted moving object as the trackingtarget candidate.

[Supplementary Note 10]

The monitoring system according to any one of supplementary notes 1 to9, in which when detecting the predetermined action, the mobile terminalshortens an interval for acquiring the position information as comparedwith a case in which the predetermined action is not detected.

[Supplementary Note 11]

The monitoring system according to supplementary note 7, in which whenthe tracking unit determines the tracking target, the management devicechanges an interval at which the management device acquires the positioninformation to an acquisition interval for the case in which thepredetermined action is not detected.

[Supplementary Note 12]

The monitoring system according to any one of supplementary notes 2 to11, in which when accepting an input to the mobile terminal, the firsttransmission unit transmits the position information of the mobileterminal to the management device at the predetermined interval.

[Supplementary Note 13]

The monitoring system according to any one of supplementary notes 2 to11, in which

the mobile terminal further includes an acceleration sensor, and

the first transmission unit transmits the position information of themobile terminal to the management device at the predetermined intervalwhen acceleration acquired by the acceleration sensor is equal to orgreater than a predetermined value.

[Supplementary Note 14]

A facility monitoring system including:

a mobile terminal held by a security guard who monitors a specificfacility;

a management device which communicates with the mobile terminal andmonitor the specific facility; and

at least one camera capable of photographing at least a part of thespecific facility, in which

the mobile terminal includes a first transmission unit which transmitsposition information of the mobile terminal to the management device ata predetermined interval, and

the management device includes a control unit which controls, whenacquiring the position information, a direction of the camera based onthe position information.

[Supplementary Note 15]

A mobile terminal including:

a first transmission unit which transmits position information of theown terminal to a management device at a predetermined interval, wherein

the first transmission unit starts transmitting the position informationto the management device when detecting a predetermined action.

[Supplementary Note 16]

The mobile terminal according to supplementary note 15, in which thefirst transmission unit transmits the position information to themanagement device at the predetermined interval when accepting an inputto the mobile terminal.

[Supplementary Note 17]

The mobile terminal according to supplementary note 15 furtherincluding:

an acceleration sensor, in which

the first transmission unit transmits the position information to themanagement device at the predetermined interval when accelerationacquired by the acceleration sensor is equal to or greater than apredetermined value.

[Supplementary Note 18]

A management device including:

an acquisition unit which acquires position information of a mobileterminal at a predetermined interval; and

a control unit which controls, when acquiring the position informationof the mobile terminal, a direction of a camera based on the positioninformation.

[Supplementary Note 19]

The management device according to supplementary note 18, in which thecontrol unit controls the direction of the camera based on the positioninformation of the mobile terminal when receiving control switchinginformation for switching camera control from the mobile terminal.

[Supplementary Note 20]

A monitoring method including:

acquiring position information of a mobile terminal at a predeterminedinterval; and

controlling, when the position information is acquired, a direction of acamera based on the position information.

[Supplementary Note 21]

A monitoring program for causing a computer to execute processes of

acquiring position information of a mobile terminal at a predeterminedinterval; and

controlling, when the position information is acquired, a direction of acamera based on the position information.

[Supplementary Note 22]

A monitoring system including:

a mobile terminal; and

a management device which communicates with the mobile terminal, inwhich

the mobile terminal includes a first transmission unit which transmitsposition information of the mobile terminal to the management device,and

the management device includes:

a control unit which controls, when acquiring the position information,a direction of a camera based on the position information;

a tracking unit which extracts one or more tracking target candidatesbased on an image/images acquired from the camera; and

a display unit which displays the image/images and highlights thetracking target candidates included in the image/images so as to beselectable.

[Supplementary Note 23]

The monitoring system according to supplementary note 22, in which thetracking unit determines a selected candidate as a tracking target andtracks the tracking target.

[Supplementary Note 24]

The monitoring system according to supplementary note 22 or 23, in whichthe tracking unit identifies a holder of the mobile terminal based onthe image/images acquired from the camera and tracks the identifiedholder as a provisional tracking target.

[Supplementary Note 25]

The monitoring system according to any one of supplementary notes 22 to24, in which

the tracking unit further extracts a moving object different from theholder of the mobile terminal based on the image/images acquired fromthe camera, and

the display unit highlights the extracted moving object as the trackingtarget candidate.

This application claims priority based on Japanese Patent ApplicationNo. 2015-052795 filed on Mar. 17, 2015, the disclosure of which isincorporated herein in its entirety.

INDUSTRIAL APPLICABILITY

The present invention is applicable to a monitoring device, a monitoringmethod, a monitoring program, a monitoring system, and a facilitymonitoring system which perform monitoring.

REFERENCE SIGNS LIST

-   1000, 2000 Monitoring system-   1100, 2100 Mobile terminal-   1110 Position acquisition unit-   1120, 2120 Motion detection unit-   1130 Communication unit-   1131 First communication unit-   1132 Second communication unit-   2140 Acceleration sensor-   1200, 2200 Management device-   1210 Communication unit-   1211 First communication unit-   1212 Second communication unit-   1220, 2220 Control unit-   1221, 2221 Tracking unit-   1222 Storage-   2223 Selection unit-   1230 Display unit-   300-1 to 300-N Camera-   100, 200 CPU-   130 Display unit-   110, 210 Input accepting unit-   120, 220 Memory-   230 Connection unit

1. A monitoring system comprising: a mobile terminal; and a managementdevice configured to communicate with the mobile terminal, wherein themobile terminal includes a first transmission unit implemented at leastby a hardware including a processor and configured to transmit positioninformation of the mobile terminal to the management device at apredetermined interval, and the management device includes a controlunit implemented at least by the hardware and configured to control,when acquiring the position information, a direction of a camera basedon the position information.
 2. The monitoring system according to claim1, wherein the control unit controls the direction of the camera to aposition indicated by the position information.
 3. The monitoring systemaccording to claim 1, wherein the mobile terminal includes a secondtransmission unit implemented at least by the hardware and configured totransmit, to the management device, control switching information forswitching control of the camera when detecting a predetermined action,and the control unit controls, when receiving the control switchinginformation, the direction of the camera based on the positioninformation of the mobile terminal.
 4. The monitoring system accordingto claim 3, wherein the management device has a normal mode formonitoring a predetermined range and a tracking mode for performingmonitoring based on the position information, and the control unitswitches, when receiving the control switching information, the normalmode to the tracking mode and controls the direction of the camera basedon the position information of the mobile terminal.
 5. The monitoringsystem according to claim 1, wherein the control unit acquires positioninformation of each camera, and the control unit selects at least onecamera to be controlled from the camera based on the acquired positioninformation of the camera and the acquired position information of themobile terminal.
 6. The monitoring system according to claim 1, whereinthe management device includes: a tracking unit implemented at least bythe hardware and configured to extract one or more tracking targetcandidates based a image/images acquired from the camera; and a displayunit implemented at least by the hardware and configured to display theimage/images and highlight the tracking target candidates included inthe image so as to be selectable.
 7. The monitoring system according toclaim 6, wherein the tracking unit determines the selected candidate asa tracking target and tracks the tracking target.
 8. The monitoringsystem according to claim 6, wherein the tracking unit identifies aholder of the mobile terminal based on the image/images acquired fromthe camera, determines the identified holder as a provisional trackingtarget, and tracks the provisional tracking target.
 9. The monitoringsystem according to claim 6, wherein the tracking unit further extractsa moving object different from the holder of the mobile terminal basedon the image/images acquired from the camera, and the display unithighlights the extracted moving object as the tracking target candidate.10. The monitoring system according to claim 1, wherein when detectingthe predetermined action, the mobile terminal shortens an interval foracquiring the position information as compared with a case in which thepredetermined action is not detected.
 11. The monitoring systemaccording to claim 7, wherein when the tracking unit determines thetracking target, the management device changes an interval at which themanagement device acquires the position information to an acquisitioninterval for the case in which the predetermined action is not detected.12. The monitoring system according to claim 2, wherein when acceptingan input to the mobile terminal, the first transmission unit transmitsthe position information of the mobile terminal to the management deviceat the predetermined interval.
 13. The monitoring system according toclaim 2, wherein the mobile terminal further includes an accelerationsensor, and the first transmission unit transmits the positioninformation of the mobile terminal to the management device at thepredetermined interval when acceleration acquired by the accelerationsensor is equal to or greater than a predetermined value. 14.-17.(canceled)
 18. A management device comprising: an acquisition unitimplemented at least by a hardware including a processor and configuredto acquire position information of a mobile terminal at a predeterminedinterval; and a control unit implemented at least by the hardware andconfigured to control, when acquiring the position information of themobile terminal, a direction of a camera based on the positioninformation.
 19. The management device according to claim 18, whereinthe control unit controls the direction of the camera based on theposition information of the mobile terminal when receiving controlswitching information for switching camera control from the mobileterminal.
 20. A monitoring method comprising: acquiring positioninformation of a mobile terminal at a predetermined interval; andcontrolling, when the position information is acquired, a direction of acamera based on the position information. 21.-25. (canceled)